An augmented reality (AR) system and/or a virtual reality (VR) system may generate a three-dimensional (3D) immersive augmented and/or virtual reality environment. A user may experience this virtual environment through interaction with various electronic devices. For example, a helmet or other head mounted device including a display, glasses, goggles and the like that a user looks through, either when viewing a display device or when viewing the ambient environment, may provide audio and visual elements of the virtual environment to be experienced by a user. A user may move through and interact with virtual elements in the virtual environment through, for example, hand/arm gestures, manipulation of external devices operably coupled to the head mounted device, such as for example handheld controllers, gloves fitted with sensors, and other such electronic devices. Physical separation of the external device (at which a user may input various selections to be carried out in the virtual environment) from the head mounted device in which the virtual environment is displayed, may cause inconsistency in how the input is executed in the virtual environment.